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ABSTRACT 


Several  methods  of  combining  a  number  of  time  series  into  a  single  series  are 
discussed.  They  are  all  individual  filtering  followed  by  summation  and  are  somewhat 
like  Wiener  filtering  in  that  a  least  squares  criterion  is  used  to  define  the  filter  co¬ 
efficients.  They  differ  from  Wiener  filtering  in  that  signal  information  is  given  in  the 
form  of  various  linear  constraints  on  the  filter  coefficients  rather  than  being  given  as 
a  signal  correlation  function.  The  formulas  are  worked  out  explicitly  for  the  case  of 
two  time  series  and  three  filter  points  and  presented  in  such  a  way  as  to  make 
generalization  clear. 


Accepted  for  the  Air  Force 

Franklin  C.  Hudson 

Chief,  Lincoln  Laboratory  Office 


DEFINITIONS 


Let  fj(T)  be  an  ordinary  sampled -data  filter  on  an  x^(t)  input  time  series  and 
let  likewise  on  x^(t).  The  final  output  y(t)  is  the  sum  of  the  separate  outputs. 


Mathematically  we  have 

2 

y(0  =  S  S 

k=l  T=0 

For  the  sake  of  illustration,  we  will  use  only  n  =  2  time  filters  and  2  time 
series  and  we  will  further  choose  three  px)int  filters,  i. e. ,  1=3.  Results  will  be  pre¬ 
sented  so  that  generalization  should  be  clear  to  the  reader.  Formula  (1)  may  be  written 
explicitly  as 

y(t)  =  [  x^(t)  filtered  ]  +  [  X2(t)  filtered  ] 


(2) 


Illustration  #1  Minimum  variance  (power) 


It  is  desired  to  find  the  filter  coefficients  which  will  minimize  the  pxjwer  in  the 


filter's  output. 


P  =  power 


(3) 


t 


1 


If  the  filter  coefficients  are  zero,  the  output  power  will  be  zero  so  the  problem 


is  trivial  and  not  fully  formed.  Later  we  will  introduce  constraints  which  bring  about 
more  interesting  solutions.  However,  it  is  worthwhile  going  through  with  this  prob¬ 
lem  as  it  contains  the  basic  mathematics  in  simplest  form.  The  situation  can  be  com¬ 
pared  with  Laplace's  equation  where  the  solution  is  a  constant  until  boundary  values 
are  considered. 

The  power  is  a  quadratic  function  of  the  filter  coefficients.  To  find  the  minimum 
we  treat  each  coefficient  as  a  variable  and  set  partial  derivatives  of  power  v/ith  respect 
to  the  filter  coefficients  equal  zero.  Because  of  this  the  method  is  called  least  squares. 
This  will  give  a  set  of  n  x  =  6  equations  for  the  six  unknowns. 

For  example,  to  get  one  equation  we  insert  (2)  into  (3)  and  differentiate  with 
respect  to  f^(0). 


(4) 


(5) 


where  we  navc  introduced  the  definition  of  correlation 

=  Z  X^(t+T)  (6) 

If  the  data  x(t)  is  imagined  to  have  infinite  duration  in  time,  it  is  necessary  to 
introduce  a  normalizing  factor  into  (6).  In  practice  it  is  unimportant. 


Likewise  differentiating  with  respect  to  each  of  the  other  six  coefficients  of  f 
gives  six  equations  which  may  be  arranged  as: 


rii(O)  rj2(0)  r^^C-I)  rj2(-l)  ■  rjj(-2)  rj2(-2) 

fj(0) 

0 

r2i(0)r22(0)  1)  r22(- 1) 

^  \  .  .  ..  _  _  _ 

f2(0) 

0 

ri,(l)  rj2<l)  rj^(O)  rj2(0)  rjj(-l)  r^2(-l) 

1 

fj(l) 

0 

f2(l) 

0 

rii(2)ri2(2)  r^^(l)  |  ^n^O) 

f/2) 

0 

^21^^^  ^22^^^  ^21^^^  ^22^^^  1^21^^^  ^22^^^ 

f2(2) 

0 

We  abbreviate  this  set  as 


RF  =  0 

If  we  take  the  6x6  matrix  of  coefficients  and  partition  it  into  a  3  x  3  matrix  of  2  x  2 
submatrices,  we  have 


R  = 


R(0) 

R(-l) 

R(-2) 

R(l) 

R(0) 

R(-l) 

R(2) 

R(l) 

R(0) 

(8) 


where  R(t)  is  the  2X2  matrix 


R(t)  = 


rii(T)  rjjM 
r2j(T) 


T 

=  R  (-T) 


(9) 
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It  may  be  noticed  that  all  the  submatrices  on  the  main  diagonal  are  identical  to  each 

T 

other.  The  same  may  be  said  of  other  diagonals.  Furthermore,  R(t)  =  R  (-t)  By 

inspection  the  matrix  is  symmetric.  The  matrix  is  positive  semi-definite  because 

T 

the  power  output  of  the  filter  is  a  quadratic  function  of  the  f  coefficients  F  RF,  which 
must  be  positive  for  any  F.  All  this  is  the  Toeplitz  property  of  R. 

The  solution  to  this  problem  is  that  all  the  filter  coefficients  equal  zero  because 
the  right  side  of  Equation  (7)  equals  zero,  as  we  anticipated.  The  importance  of  this 
example  comes  thru  the  introduction  of  the  Toeplitz  matrix  which  plays  a  central  role 
in  time  series  analysis  and  the  examples  to  follow. 


Illustration  #2  The  constraint  f  j^(O)  =  1 


V'  2 

Suppose  we  wish  to  minimize  ^  y(t)  as  before  but  to  make  the  additional 

t 

constraint  f ^^(0)  =  1.  One  method  would  be  to  proceed  as  before  replacing  f ^^(0)  by  1 


The  trouble  with  that  approach  is  that  we  will  no  longer  have  a  Toeplitz  matrix 
like  that  in  formula  (7)  but  rather  a  matrix  with  much  less  symmetry.  For  this  reason 
we  will  introduce  the  method  of  Lagrange  multipliers.  It  will  still  give  us  a  Toeplitz 
matrix  like  (7)  for  a  large  class  of  problems  with  linear  and  quadratic  constraints. 
Several  problems  of  this  class  will  be  discussed  in  later  examples. 

The  Lagrange  method,  as  applied  to  illustration  #2,  is  to  minimize 


(10) 
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by  setting  it's  derivatives  with  respect  to  each  of  the  six  coefficients  of  f  and  with 
respect  to  X  equal  to  zero.  Then  solve  the  seven  equations  for  the  f's  and  X. 

9E 

To  see  that  the  Lagrange  method  gives  the  desired  result,  note  that  gives 

o  \ 

the  constraint  equation.  Then  at  the  minimum  of  E,  X  will  multiply  [fj^(O)  —  1  ]  =0. 

Thus  minimizing  E  is  equivalent  to  minimizing  ^y(t)^. 

t 

Partial  derivative  of  E  with  respect  to  the  f  coefficients  gives 


X 

0 

R 

F 

= 

0 

0 

0 

0 

If  we  divide  F  by  X  on  the  left  side  and  X  by  X  on  the  right,  we  will  have  a  set  of 
simultaneous  equations  for  F/X.  Then  we  can  pick  f^(0)  =  1  and  this  determines  X  and 
all  the  f  coefficients. 


Illustration  #3  Levin's  filter 

VT  2 

Again  we  minimize  y(t)  but  with  the  constraints 
1  =  f^(0)  +  f2(0) 

0  =  f^(l)  +  f2(l)  (12) 

0  =  f^(2)  +  f2(2) 
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Levin  arrived  at  these  constraints  by  wanting  the  filter  to  pass  that  signal 
which  is  common  to  Xj^(t)  and  X2(t)  without  distortion.  To  show  there  is  no  distortion 
of  the  common  part  s(t)  of  Xj^(t)  and  X2(t)  write 


x^(t)  =  s(t)  +  z^(t) 
x^(t)  =  s(t)  +  z^(t) 


(13) 


Then  from  (2) 


y(t)  =  s(t)  +  (terms  in  and  z^  but  not  s) 
Using  Lagrange’s  method,  we  differentiate 
.2^ 


E  =  {  Eyft)  }  -)>jlfi(0)+f2(0)-ll  -X2lti<l)  +  f2(l)l  -X3lfj(2)  +  f2<2)l  (14) 


getting  for  ^ 


—  — 

f^(0) 

1^(0) 

f/1) 

^2 

R 

f2(l) 

^2 

f^(2) 

^3 

f2(2) 

1 

Co 

1 _ 

(15) 


which  we  may  write  as 


6 


-  — 

“  - 

1 

0 

0 

1 

0 

0 

h 

0 

1 

0 

R 

F 

= 

z 

0 

1 

0 

0 

0 

1 

0 

0 

1 

_  _ 

_  _ 

which  we  abbreviate  as 

RF  =  GA 


(16) 


(17) 


We  can  write  the  constraint  equation  down  directly 


taking 


9E 


=  0  for  each  X. 


or  go  through  the  formalism  of 


f/0) 

1 

1 _ 

f2(0) 

1  1  0  0  0  0 

f/1) 

1 

0  0  110  0 

f2(l) 

= 

0 

0  0  0  0  1  1 

f^(2) 

0 

f2(2) 

which  we  may  abbreviate  as 


(18) 


T 

G  F  =  D 


(19) 
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Now  we  want  to  solve  the  set  (17)  and  (19) 


T  T  -1 

G  F  =  G  R  GA 

T  -1 

multiply  (17)  by  G  R 

T  - 1 

D  =  G  R  GA 

insert  (19) 

T  -1  -1 

A  =  (G  R  G)  D 

solve  for  A 

(20) 

\ 

RF  =  G(G^r‘^G)’^D 

insert  into  (17) 

» 

-1  T  -1  -1 

F  =  R  G(G  R  G)  D 

solve  for  F 

(21) 

> 

which  is  the  desired  solution.  From  (21)  it  may  be  seen  that  a  scale  factor  in  R  does 
not  affect  F.  This  justifies  the  earlier  assertion  that  a  scale  factor  in  R  is  unimportant. 
The  actual  power  minimum  obtained  is 

T  T  T  “1  ‘1 

F  RF  =  D  (G  R  G)  D  (22) 

-1  2 
The  quantity  R  G  is  computed  efficiently  by  the  Levinson  recursion  . 

Repositioning  the  one  in  the  vector  D  will  produce  Levin  filters  with  delayed 

T 

signal  outputs.  Some  positions  will  reduce  the  output  noise  power  F  RF  more  than 
others. 


4 
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Illustration  #4  A  modification  of  Levin's  constraint 


In  one  application  of  Levin’s  filter  it  was  observed  that  the  filter  coefficients 
weighted  some  channels  very  weakly  or  even  with  negative  weights.  This  seepied  un¬ 
satisfactory  because  in  the  experimental  data  the  channel  gains  were  not  well  controlled 
and  the  signals  s(t)  were  not  truly  identical  in  the  channels.  A  modification  of  Levin's 
constraints  which  retains  the  idea  of  passing  the  common  part  of  Xj^(t)  and  X2(t)  without 
distortion  but  leads  to  an  even  weighting  of  signals  is 


1/2  =  f^(0) 

1/2  =  f2(0) 

0  =  fj(l)-Hf2(l) 

0  =  fj^(2)  +  f2(2) 

These  constraints  lead  to  the  same  formal  equations  as  before 


RF  =  GA 
T 

G  F  =  D 

where  in  this  case 


G  = 


10  0  0 
0  10  0 
0  0  10 
0  0  10 
0  0  0  1 
0  0  0  1 


D  = 


1/2 

1/2 

0 

0 
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(23) 


(17) 

(19) 


(24) 

(25) 


Although  signals  are  forced  to  be  weighted  equally  some  channels  may  still  be 
weighted  more  heavily  than  others,  due  to  other  filter  lags  than  the  zeroth  lag.  Although 
this  might  be  good  it  led  to  trouble  in  one  application  when  experimental  data  channel 
timing  was  not  well  controlled. 


Illustration  #5  Prediction  Error 

The  constraints 


fj(0)  =  1 

f2(0)  =  0 


(26) 


give  rise  to  the  prediction  error  filter  at  unit  span  for  the  signal  Xj^(t).  This  filter  is 
so  named  because  (2)  can  be  written 


y(t)  =  Xj^(t)  —  (terms  dependent  on  time  before  t) 


(27) 


Since  y(t)  is  minimum,  the  "terms  dependent  on  time  before  t"  try  to  cancel 
t 

Xj^(t).  Those  "terms"  are  the  prediction  filter  of  x^(t)  and  y(t)  is  the  prediction  error 
signal. 


The  prediction  error  filter  for  span  =  2  is  given  by  the  constraints 


=  1 
f^CO)  =  0 
fj(l)  =  0 
fjd)  =  0 


(28) 


Span  =  2  means  the  prediction  is  done  with  terms  depending  on  time  at  and  before  t-2. 
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Illustration  #6  The  general  linear  constraint 
A  fully  generated  example  of  two  linear  constraints  is: 


This  leads  to 


G  = 


10 

w 

10 

20 

^20 

11 

'"ll 

21 

W 

21 

12 

w 

12 

22 

w 

22 

D  = 


Illustration  #7  Quadratic  constraints 

Quadratic  constraints  have  not  become  well  known  but  they  are  tractable  using 
similar  methods. 

w. 

This  will  be  illustrated  by  means  of  the  annihilation  filter.  *  The  constraint  is 
►  that  the  power  coming  out  of  the  f ^(t)  filter  plus  the  power  coming  out  of  the  f2(T)  filter 


*  It  utilizes  coherencies  among  the  channels  to  annihilate  the  output  y(t). 
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should  have  a  constant  (  =  1)  expected  value  but  there  should  still  be  minimum  power  in 
the  sum  (3)  as  described  in  Illustration  #!• 

Explicitly,  this  constraint  is 

1  =  Zlf/O)  x^(t)  +  fj(l)  Xj(t-l)  +  f  j(2)  x^(t-2)]^  +  [f2(0)  ^^(t)  +  £2(1)  X2(t-1)+  £2(2)  X2(t-2)]^ 
t 

Taking  derivatives  as  in  example  #1  leads  to 


r 

— 

rij(O) 

0 

1 

0 

1 

0 

0 

1  0 

1 

^22^'^^ 

1  0 

^22^'^^ 

J  R-X 

r,i(l) 

0 

1 

1 

0 

j 

0 

0 

122(1) 

1  ° 

'22®> 

1  ° 

r22(-2) 

rii(2) 

0 

0 

rii(O) 

0 

0 

r22(2) 

1  0 

^22^^^ 

1  “ 

- 

1 

1 

- 

J 

This  is  a  generalized  eigenvector -eigenvalue  problem  whose  solution  may  be 
obtained  by  standard  methods.  A  recursion  scheme  to  speed  computations  like  the 
Levinson  method  has  not  yet  been  developed  for  problems  with  quadratic  constraints. 

JC/lmm 
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